Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning

Modern services consist of interconnected components, e.g., microservices in a service mesh or machine learning functions pipeline. These can scale and run across multiple network nodes on demand. To process incoming traffic, components have to be instantiated traffic assigned these instances, taking capacities, changing demands, Quality Service (QoS) requirements into account. This challenge is usually solved with custom approaches designed by experts. While this typically works well for the considered scenario, models often rely unrealistic assumptions knowledge that not available practice (e.g., priori knowledge). We propose DeepCoord, novel deep reinforcement approach learns how best coordinate geared towards realistic assumptions. It interacts relies available, possibly delayed monitoring information. Rather than defining complex model an algorithm achieve objective, our model-free adapts various objectives patterns. An agent trained offline without expert then applied online minimal overhead. Compared state-of-the-art heuristic, DeepCoord significantly improves flow throughput (up 76%) overall utility (more 2x) real-world topologies traces. also supports optimizing multiple, competing objectives, respect QoS requirements, generalizes scenarios unseen, stochastic scales large networks. For reproducibility reuse, code publicly available.